DOCOHENT BESOHE 



ID 104 669 

ADTHOR 
TITLE 
POB DATE 
NOTE 



BDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



SE 018 795 

Nelson, ttiies A. X 
A New Diaension in Selecting Science Teachers. 
Har 75 

33p.; Paper presented at the annual RGetiug of the 
National Association for Research in Science Teaching 
(48th, Los Angeles, California, March 1975) 

MF-S0.76 HC-$1.95 PLUS POSTAGE 

Classroom Techniques; *Credentials; Educational 

Research; Higher Education; Science Education; 

♦Science Teachers; *Teacher Behavior; Teacher 

Characteristics; *Teacher Placement; Teacher 

Qualifications; *Teacher Selection; Teaching 

Skills 

Research Reports 



ABSTRACT 

This work addresses itself to improving the procress 
of teacher selection by providing more information about teaching 
ability. The study reported was undertaken to determine the effects 
of incorporating into the educational placement credentials a set of 
materials relating to a science teacher's classroom performance. In 
the report, past research is examined^ for clues to credential 
usefulness and improvement. A detailed description of the new 
credential is provided, as well as the results of an evaluative study 
to determine its effectiveness.. Economical, technical, legal, and 
ethical issues relating to tbe new credential were examined in the 
study but not reported in this paper. Related research is presented 
showing various types of selection processes and how unrelated most 
are to actual teaching performance. To determine the usefulness and 
effectiveness of each part of the credential package, seven separate 
prototype credentials were developed. Examples are presented as well 
as a description of the evaluation process. (Author/EB) 
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Selection o£ teachers is a fascinating subject, well worth the coverage 
it receives. This present work addresses itself, toward improving the 
selection process, by providing more information about teaching ability. 
Educational recruiters have often con5)lained that placement credentials 
provide little infoiraation about a candidate's ability to teach. This 
study was undertaken to determine the effects of incorpora'cing into the 
educational placement credentials a set of materials relating to a science 
teacher's classroom performance. 

In this report, past research is examined for clues to credential usefulness 
and iirprovement. A detailed description of the new credential is provided 
as well as the results of an evaluative -study to determine its effective- 
ness. Economical, technical, legal, and ethical issues relating to the new 
credential were also examined 'but are not reported here. 



Related Research 

^^LxA has been written for potential enployers on how to select teachers. 
Unfortunately most of it is based on opinion, \*ile little research has 
been conducted. This review will draw from research v;hich, in the author's 
opinion, has a bearing on development of a placement credential better 
, representing a candidate's "ability to teach". 

Gilbert (1966) , after conqpleting a national survey of teacher selection 
processes in large ptiblic school systems, was dismayed at those used. 
He concluded that "selection methods focused exclusively on the overt and 
peripheral aspects of teachers, and appeared to be governed by what was 
most easily obtained rather than on what might be important to assess. 

Selection methods have a direct effect on final decisions, and Scott (1964) 
has polarized the methods used. A clinical method, he notes assumes both 
job and teacher are too complex for investigation and that different teach- 
ing approaches lead to coinparable end results. In contrast, an actuarial 
method assumes predictors of teaching success exist and should be used. 

In an earlier work Toops . (1945) noted five, selection methods \Aich could 
be corjsidered actuarial in nature. First, a sunHnation of characteristic 
scores organizes bits of information into categories which are 'sunmed' 
by some process. In a second method, successive hurdles, items with 
high validity are applied until the desired nunber of candidates is reached. 
Identification of certain characteristics deemed essential for the job is 
a third method and results in a precise profile of candidates.' Fourth, an 
off-shoot of this method results in the establishment of an ideal profile; 
a candidate who diverges least from the ideal is rated the most desirable. 
Finally, the outstanding merit method gives the most weight to candidates 
who possess outstanding qualifications in more than one cliaracteristic. 



Rather than describe methods of selection, some witers have identified 
discreet stq-; within methods- Based on an extensive research study, 
Bolton (1970) suggests seven stages in the selection process. One step, 
gathering data about candidates, has implications for developing place- 
ment credentials. Bolton makes several reccxnmendations : 1) avoiding letters 
of recomnaidation as accurate appraisals of character or potential worth; 
2) systematically assessing teaching performance; 3) seeking information 
from a variety of sources; 4) seeking information regarding the conditions 
under which a reference has observed the candidate; 5) using a single 
summary document. ' ^ *>' 

Reviewing steps in the selection process Bauthues (1968) notes that most 
interviews and reference letters are unreliable and invalid, and recom- 
mends a standard foim for both in order to increase reliability. He also 
recoiranends that predictive validity studies be conductai iji each school 
district in order to establish better selection criteriC^ .Medley (1967) , 
Redfem (1967), Majetic (1972), Masonis (1972), and Mitzel (1967), all suggest 
similar validity studies. 

There are a variety of selection jnethods and^J^ gather data about candidates 
More research must be done to establish reliable" arid val^d selection criteria.^ 
Specifically more reliable and standardized letters of reference could aid 
in the selection process. Since letters of reference axe only one aspect 
of most placement credentials, what other items should be added to, or 
deleted from, a placanent credential? Very few surveys, published in the 
literature, have been conducted to determine Item usefulness. 

Paquette (1966) surveyed employers and college placement' personnel to 
determine what should be included in a credential, o!E what value the 
credential was, and ivhat factors were important in the selection process. . 
It should be notod that this studbyr was concerned with placement credentials 
for business and industry. His findings indicated that employers and - 
college personnel agreed that personal data, educational history, and work 
experience should be included in a credential. The agreement between 
the two groups differed significantly in the area of letters of reference. 
College personnel believed faculty and mployer references were quite 
important ranking them, respectively, fourth and fifth in what should be 
includd^, while enrdoyers ranked these same items sixth and twelfth. 
Employers ranked a recruiter's report as the morjt important factor in the 
selection process, while student credentials were ranked' fourth, faculty 
references fifth, and past employers' references eighth. Based on his 
findings, Paquette, too,, recommended that personal references "be eliminated 
from the credential. ' i 

Dropkin and Castiglione (1969) studied the usefulness of itans in teacher 
placement credentials. Using a 42 item questionaire they sought to deter-, 
mine items considered indispensible by personnel recruiters; if a relation- 
ship existed between school district size and item usefulness ; and if 
a relationship existed between usefulness of an iton and the difficulty 



I 



in obtaining teachers. ^In their report the itans considered indispensible 
for inexperienced teachers were letters o£ reference from cooperating 
teachers and supervisors, location and type of student teaching, under- 
graduate transcript, and graduate courses vdth grades. These results 
coupled with similar findings for experienced teachers led them to suggest 
that itdns ranking highest in importance were concerned -with the candid- 
ate's most recent observed teaching behavior, while items ranking lowest 
in iiiportance dealt with the candidate's cjraracter. DrOTkin and Casti- 
glione also concluded ^bat rated items of importance haa a negligible 
"relationship to school district size and difficulty in obtaining teachers. 
An inference from- this information could be that any addition to a place- 
ment credential should be more directly related to the candidate's teach- 
ing performance. 

One research study, not directly related to placement credentials, but 
which has implications for developing useful credentials, was that done 
'^by Bolton "(1968). His study investigated the effects of four variables on 
the selection process; instructions on how to process-information, number 
of documents describing the candidate, masking of selected information, and 
interview information. The dependent variables used were the total time 
taken to process the information and how well, how certain, and consis- 
tently a group could distinguish between candidates. Teacher selection ^ 
materials developed included written documents such as transcripts, 
credentials, etc.; audio and audio visual materials .for simulating 
teacher interview situations; and- job descriptions. These materials 
were given to 144 school principals who were to select fictitious can- 
didates for the jobs. Optimxmi res.ults as far as time, discrimination, ^ 
consistency and degree of certainty occurred when principals 'were given 
instructions on how to process the information; when a single suirenary 
document was used; when no information regarding the candidate^ was 
masked; and when interviews included visual ^os well^as audio stimuli. 

Bolton's study appears to be the first to use a filmed interview with a 
candidate. Although he used it to standardize an interview in a ccn* 
trolled e)q)eriment, others have utilized the idea as a placement function. 
The University of Maine video tapes an unrehearsed, unedited conversation % 
between the candidate and an interviewer. This 15 minute video tape along 
with a personal resume is sent to an eirployer so that a decision about 
further negotiations can be made. Lang (1970) makes available to pro- 
spective employers a 5-10 minute segment of a teacher candidate's class- 
rocmi performance. He believes that selected editing of the video tape is 
permissible since the resume i^^ naturally favorable to the candidate. 
He notes that recruiters who have used the video tapes liked them very 
much. The Universit)' of New Mexico Placement Bureau (1971) dubs several 
2-3^*minute segments on one video tape of several candidates for the same 
teaching position. This is iJone at the request of a recruiter ^who 
describes to the placement director the type of position he has ^vail- ' 
able. The resulting tape presents all of the available candidates on a 
single half- hour tape. To date the writer has found no published research 
on the effect or usefulness of a filmed/video taped teaching performance. 
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Summary 



The literature review has sho\<n various types of selection processes 
arid how unrelated most are to actual teaching performance. It is clear 
there is a need for reliable and valid infomiation suitable for pre- 
dicting job success- Descriptive materials which are standardized, provide 
comparable data and are related to a job description appear to be most 
useful in the selection process. A visual representation of the candidate , 
• .in interview of job simulation is a potential, but unresearched resource 

^£or placement purposes^ Presumably any addition to or change in a place- 

^ menf' credential should be guided by this information. 

" Development of the Credential 

The initial effort to improve the placement credoitial was directed 
toward the letter of reference. The previous section has shown that 
several researchers reconmiend its elimination and/or standardization. 
Anyone >/ho has written or read such letters can infer the reasons why. It 
is difficult to communicate in a few short paragraphs a candidate's teaching 
ability. Many reference writers describe moral or personal characteristics 
believing these to be indicative of good teaching; others describe courses 
taken by the candidate and grades received; in either case little of the 
candidate's teaching ability is presented. Researchers (Bolton 1970) 
(Bauthues 1968) have recommended a standard form which would increase 
reliability and make comparisons possible. 

At the end of the practicum experioice, science cooperating teachers for the 
UW-MSN are given a form which develops a profile for the candidate in six 
/areas: subject matter mastery, planning, teaching ability, classroom climate, 
y evaluation and personal characteristics* , These six areas were made the 
fj basis for writing the author's letters of reference. Under each heading a 
V/ brief description of the candidate's abilities in that area was piovided. 
Prefacing -eacli letter was a brief description of the setting in whidi the 
candidate was observed. Ending eadi letter was a prediction of the candidate's 
success, based on the observations made j.n the six areas. In the appendix 
is an example of the reference letter. This type' of reference letter was 
thou^t to provide the reader with a frame of reference, ^^ome subjective 
information about the candidate's ability as a teacher, and a prediction of 
success. * Standardizing' the letter for candidates, i.e., comparing the 
candidates in the same six areas, would make comparisons possible. 

' The next addition to the placement credential was a list describing the 
objectives achieved while in the methods course and pr3cticum experience. 

! Since the science methods course* and practicum experience is based on 
64 objectives directly related to scioice teaching, many of them being 
based on classroom performance, it was thought that. such a list would 
provide more information about the candidate's teachin:^ ability. A 
summary sheet, found in the appendix, was developed doi>cribing how 
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candidates achieve objectives, what the emphasis for each objective was, 
and how well 'the objective was achieved. Objectives achieved in the 
methods cours<e are evaluated by tlie methods professor, while those 
completed in the practicum are evaluated by the -cooperating ^teacher. 

Because most of the objectives attained by each candidate are elected, * 
the summary sheet described above does not provide a comprehensive 
description of the candidate's teaching ability. A supplementary source 
was provided in the form of a 'pictograph' obtained from the rating form, 
completed by the cooperating teacher.;' and described 'above (See Appendix)-. 
This pictograph shows how the rater believes the candidate compares to 
other beginning teachers and to himself in six areas subject matter, 
mastery, planning, teaching ability, classroom climate, evaluatiqn* and 
personal characteristics. Two pictographs were provided, one completed by 
the cooperating teacher who has observed in depth several (1-15) candidates, 
and one completed by the science methods professor/student teacher 
supervisor who has observed many .(300) candidates. These pictographs, 
in essence, are a single summary document making possible coii5)arisons 
between the candidate and other beginning teachers whom the rater has 
observed. 

The final addition to the placement credential was the addition of a 15- 
25 minute sample of the candidate's teaching on a video tape. This 
sample was selected from 1 to 2 hours of video tape made during the 
candidate's last, two weeks in the practicum experience. A sincere effort 
was made to show the candidate in several different teaching situations 
discussing, lecturing, and tutoring. These teaching sequences were taken 
by a university supervisor and those portions appearing in the placement 
credential were selected by hm. The final product was recorded on a 
three-quarter inch video cassette and opened with a visual describing the 
candidate's educational background, v^iere he student -taught, and how the 
teaching sequences were gathered. All visuals refrained from making 
evaluative statements about the candidate or in any way directing the viewer 
to note specific highlights. Enclosed with each video cassette was« a 
card briefly describing the teaching sequences and where to find them on. 
the tape. 

0 

To summarize: based on ^research findings, additions and changes were 
made in the educational placement credentials of beginning science teachers. 
These included a more standardized letter of reference; a summary sheet 
describing the objectives achieved by the candidate during the methods 
course and practicum e^qjerience; a pictograph; and a short video taped 
teaching sequence made during the candidate's last weeks in the practicum. 

V 

Evaluation of the Credential 

To determine the usefulness and effectiveness of each part of the creden- 
tial package, seven separate prototype credentials were developed (See 
Table I) . Administrators and science ?i?)ervisors throughout the -State 
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of V/isconsin, involved in hiring new teachers, were invited to participate 
in the evaiuation process. 0£ those who were invited, 32 were free on 
the evaluation days and actually participated. Six of the participants 
were science teachers or science department heads, seven were personnel 
directors., six were principals, six were assistant superintendents, and the 
remains ng seven had various titles such as Coordinator of Personnel, 
Director of Op^grations , Coordinator of Secondary Education. Participants 
were given tasks designed to provide answers to four questions: 

1. IVhat effect does the credential type have on a 
' candidate's ranking for employment? 

2. Which information in the prototypic credentials 

is most useful in ranking • candidates for employment? 

3. Which prototypic credential provides the most 
confidence in candidate ranking? 

\ 4. iVhat do potential employers look for in a video 
taped teaching sequence? 

« 

The evaluation tasks designed to obtain data on each question are shown 
in the appendix. 

* M 

Table I 

Seven Prototypic Credentials 
. Used in This Study. 

1. Written Materials * 1 

a. . Candidates personal data^ career objectives, 

avocations, etc. 

b. Course titles and grades 

c. Letter of reference from coggterating teacher 

d. Letter of reference from university supervisor 
^ e. Letter of reference from methods professor 

2. Objectives Materials 

a. * Objectives for methods course and practicum ' 

experience. \^ 

b. Pictogram which simmarizes candidates performance in 
seven ateas and is coiq)ared with other beginning 
teachers . 

3. Video taped teaching sequence, 

a. Discussion session 

b. Lecture session 

c. Laboratory session 

4. Written material and Objective material 

5. \^ritten material and Video taped teaching sequence. 
5. . Objective material and Video taped teaching sequence. 

• 7. Written and Objective Material and VAdeo taped 
teaching sequence. 
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,^ Question One 

IvTiat effect does tlie credential type have on a candidate's ranking for 
employment? 

F^othesis ^ 

Each of the prototypic credentials can be cpnceptualized as providing 
information regarding teaching ability. Further, some of the credentials 
are more representative of teaching performance than others; i.e., some 
actually display a teaching performance. From this perspective, the 
written material is least representative of actual teaching performance. 
The objective material is more representative of actual performance 
since it represents the teaching skills a candidate* has acquired and how 
well he has developed them. Finally, the video taped teaching sequences 
are most representative of performance since they actually show the 
candidate in action. Assigning a rank of one to the least representative 
credential and a three to the most representative, a ranking in terms of 
representation of performance can be obtained. (See Table. 2) 



Table 2 

Predicted Rank of Credentiail Type in. Terms" 
of Teaching Performance Representation 

Predicted level of ' 
Representation 
Type (1 low 6 high) 

Written (W) 
Objectives (0) 
Video Tape (V) 
WX) 

OV 
WOV 



]. 
2 
3 

.1+2 3 
1+3 4 
2+3 5 
1+2+3 6 



From Table 2 it was inferred that as one proceeded down the list, the 
credentials became more representative of teaching performance. 

Two hypotheses were developed: First, as the credentials became more 
representative of teaching performance, participants would agree more 
closely in ranking the candidates for desirability m employment. Second, 
as more information about the candidate's teaching ability was provided, 
the ranking assigned by a group having the same prototypic credential 
would agree more closely with the ranking assigned by individuals vAio 
had actually seen the candidates teach, i.e., university supervisors. If 
this latter hypothesis were not true, then 1) the prototypic credential 
was not representative of actual teaching performance, or 2) th6 
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participants '^-conception of what good teaching was would be different 
from that held by the individuals who actually saw the candidates teach. 
This latter alternative was explored in answering question four. 

Method " ^ ' 

Prototypic credentials were prepared for eight candidates, four in 
^iology and four in chemistry. Seven groups of four participants each 
were given the same type of credential. Participants were asked to rank 
from most to least desirable the four candidates within a subject matter 
area, using only the^material pix)vided. Each group ranked the applicants 
in both subject matter areas but using different types of prototypic 
credentials. 

To test the first hypothesis, Kendall's coefficient of concordance was 
confuted for each group to estimate how closely the four participants 
agreed in assigning ranks. These coefficients of concordance were then 
ranked, from least in agreement (lowest in magnitude) to most in agree- 
ment (highest in magnitude), and coirpared to the predicted ranking, 
using Spearman's rank correlation coefficient. If a rank correlation 
coefficient differed significantly from zero at the .05 level, the 
prediction was considered supported,* 

ft 

To test the second hypothesis, Kendall's coefficient of concordance was 
used to determine the agrement within a group, between the participants" 
average ranking and the rankings assigned by persons who had actually 
. seen the candidates teach. "The coefficients were ranked and compared 
to the predicted ranking^ using Spearman* s rank order correlation co- 
efficient. 

Results and Discussion 

The data to test the first hypothesis are presented in Tables 3 and 4. 
Inspection of these tables shows more apparent agre^ent-lji ranking the 
chemistry candidates than for the biology candidates. From the COT5)uta- 
tion of the correlation coefficient it appears that there is no relation- 
ship between a credential's representation of teaching ability and agreement 
in rariking the candidates for desirability in employment. One could easily 
conclude that the hypothesis was not supported by the data; however, 
a competing explanation is apparent from examining the directions given 
to participants who were asked to consider the candidates as applicants 
for positions in the participants own school. Since a wide variety of 
school systans were represented in'^e evaluation program, it is possible 
that no agreement could be reached. In order to control this, the 
directidns should have given the same job description to all participants. 
Feedback from participants during a debriefing session conducted after 
the evaluation session supported this point of view. Because of this the 
original hypothesis needs to be tested under more rigorous conditions 
before any conclusions regarding representativeness of teaching can be 
reached. 
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. Table 3 

Judge's Ranking of Biology Candidates 
and Groxip Coefficients of Concordance 



Prototypic 
Credential 



IVritten 



Objectives 
(0) . 



Video Tape 
CV) 



IVO 



W 



OV 



WOV 



Judge 


Candidates 


Assigned Rai 


t 

t 

U 


Purposes 




a 


b 


c 


d 


a 


4 


3 


2 


1 


b 


2 


3 


1 


4 


c 


1 


D 


4 


Z 


d 


3 


2 


1 


4 


e 


2 


3 


1 


4 


f 


2 


3 


1 


4 


S 


A 


1 
J 


1 


L 


h 


3 


2 


1 


4 


i 


2 


3 


4 


1 


• 
J 


3 


1 


2 


4 


k 


1 


2 


3 


4 


1 


1 


2 


3 


4 


m 


3 


2 


1 


4 


n' 


3 


2 


1 


4 


0 


3 


4 


1 


2 


P 


4 


2 


3 


1 


q 


3 


4 


1 


2 


r 


3 


2 


1 


4 


s 


2 


• 3 


1 


4 


t 


2 


3 


1 


4 


u 


2 


4 


1 


3 


V 


3 


4 


1 


2 


w 


2 


3 


1 


4 


X 


1 


T 

•J 


4 


2 


y 


■ 4 


3 


2 


1 


z 


I 


4 


2 


3 


aa 


2 


1 


4 


3 


bb 


3 


4 


1 


2 




,08 1.5 



.,68* 6.0 



.32 3.5 3.5 



.32 3.5 3.5 



.70* 7 



.38 5 6 



.08 1.5 7 
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correlation between predicted 
rank and actual rank .09 
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» *Signi£icantly 
different from zero 
at •05 level* 
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Table 4 



Judge Ranking of Chemistry Candidates 

and Group Coefficients of Concordance 

1^ 



Prototypic 
Credential 



1 



I 



I 



o o 

65 



Written 
0*0 



Objectives 
(0) 



Video Tape 
■ (V) 



ov 





<x 




f 


d 






o 


A 




X 

•J 


2 






b 


4 


1.5 


3 


1.5 


.83** 


6 1 


c 


3 


2 


4 










A 


0 














7 


A 

*T 








f 


3 


2 


4 


1 


1.0** 


7 2 


g 


3 


2 


4 


J 






h 




9 


A 










•J 


2 


4 








• 


2 


3.5 


3.5 




.55 


3.5 3.5 


k 


1 


2 


4 


3 






1 


1 


2 


4 


3 






m 


3 


2 


•4 


1 






n 


4 


1 


3 


2 


.55 


3.5 3.5 


0 


4 


3 . 


1 


2 






P 


4 


1 


•J 


2 






q 


3 


1 


4 


2 






r 


3 


2 


4 


1 


" .54 


1.5 5 


s 


3 


1 


4 


2 






t 


'3 


1 


4 


2 






u 


■2 


3 


4 


1 






V 


2 


1 


4 


3 


.70* 


5 6 


w 


3 


2 


4 








X 


3 


2 


4 


1 






y 


3 


1 


4 


2 






z . 


4 


2 


3 ' 


1 


.54 


1.5^ 7 


aa 


4 


' 1 


2 


3 






bb 


4 


2 


1 


3 







Correlation between predicted 
rank and actual rank -.68 



* Significantly different from 
zero at .05 level 



Tables 5 and 6 present the data gathered to test the second hypothesis. 
Cbmputa^on o£ the correlation coefficient as not significantly di-f^^erent 
from ^pfo^does not support tlie hypothesis that as more information about 
the candiilate*s teaching ability is provided, average rankings assigned 
to the candidates would agree more with the rankings assigned by univer- 
sity si?)ervisors who had observed all candidates teach* The results 
could be explained 'by competing hypotheses: as for the first hypothesis 
each participant was ranking the candidate for his own school; partici- 
pants were using a different standard for judging vSo"was the best teacher; 
or the prototypic credentials are not representative of teaching perfor- 
mance. An additional hypothesis was siqjplied by more than half of the 
participants who noted tiiat their ranking of the candidates changed depend- 
^ing oh- vrfiether the video tape was observed before the written materials 
or vice versa. These participants obtained one. ranking based on the written 
materials and then had to rerank candidates after viewing the video tapes. 
This suggests that order of presentation may be iJT?)ortant. A graphical pic- 
ture of how a candidate's rank changes with the infoimation presented is shown 
in Table 6. ' 

To review: Neither hypothesis was si^jported by the data, but an error in 
the directions to participants could explain the results. In view of this, 
judgments regarding the credentials representation of teaching ability 
will be withheld until the hypotheses are tested under more controlled 
conditions . 



Question IWo 

Which information in the prototypic credentials was roost useful in 
ranking candidates for enployment? 



Hypothesis 

Past research on credential usefulness has shown that letters written by 
individuals who have actually .observed the candidate teach, rank quite 
high (Dropkin and Castiglione, 1969; Paquette, 1966) • The cooperating 
teacher's letter of recanmendation is usually ranked niiuber one in terms of 
usefulness. Dropkin and Castiglione inferred from their surv^ that items 
receiving the highest ranking are concerned with the candidates most recent 
observed teaching behavior. From this it was predicted that those items in 
the prototypic credentials which directly relate to the candidates' teaching 
performance would rank highest. These would be the video taped teaching 
sequences and the letters of reference from cooperating teachers and 
university supervisors. Those items which would rank lowest according to 
Dropkin and Castiglione would be those which relate to the candidates' 
personal characteristics. In the prototypic credentials these items are 
candidate's personal data sheet. The predicted ranking of usefulness is 
given in Table 7. 
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coefficient of Concordance computed between a candidate s 
average ranking within a credential type and the average 
raiding assigned to candidates by persons, who observed all 
candidates • 



w 

0 
V 

wo 
wv 

ov 
wov 



•62 C3.5)* .90 C5) 

.62 (3.5) 1.00 (6.5) 

90 (7) .82 (3.5) 

.70 (5) 1.00 (6.5) 

.82 (6) ' .80 (2) 



Cbrrelation coefficient - - 21 

between actual rank and .42 
predicted rajik. 

' *Nuinbers in parenthesis indicate rank of concordance coefficient 



■Table 6 



Line graph showing candidates average ranking within a credential 
type as a function of credential type 
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Table 7 



Predicted Ranking of Credential Jtem Usefulness 



^ 



2 Video taped lecture 'sequence 

2 Video ta^eji discussion sequence 

^ 2 Video taped tutoring sequence 

5 Letter o£ reference from cooperating teacher 

5 Letter of reference from university si:5)ervisor 

5 Letter of reference from- methods professor ' 

7.5 Objec;tive li^t 

7.5 Pictograph 

9 Grade report 

10 Candidate's personal data 



After having ranked candidates from most desirable to least -desirable, 
using a particular prototypic credential, participants were asked to rank 
the usefulness of the material supplied. Participants were also asked 
if any item provided was particularly helpful in ranking, the candidates . 
If no item was particularly helpful, administrators* were asked to indicate 
what type of information should have been supplied. 

For each item in a prototypic credaitial a score was confuted from the 
assigned ranks. That item having the lowest total score was ranked one. 
The item having the next largest score was ranked second. This procedure 
was followed for the items used to rank biology candidates as well as 
those used for the chemistry candidates • Rankings of usefulness for the 
'biology and chemistry candidates were compared, using Spearman's rank order 
correlation coefficient. A significant correlation at the .05 level was 
accepted as .evidence of participant agreement of item usefulness . An overall 
ranking of item usefulness was obtained by converting all average rankings 
within a credential type to a scale of 10 and then within an item fin-^ling the 
median score. The median for each item was then ranked the lowest median 
being givoi the rank of 1 and designated as most useful, etc. 



Results and Discussion 

Average rank of itOT usefulness is displayed in Table 8. The significant 
correlation between groups of raters ranking item usefulness and using the 
same type'of credential should be pointed out. The correlation between 
predicted item usefulness and itOT usefulness as perceived by the participants 
is .78 and significantly different from zero at the .01 level, thereby 
supporting the hypothesis. During debriefing sessions participants remarked 
that the objective list and pictograph were difficult to use because they did 
not understand their purpose. After providing information relating to their 
development, participants thought such itans should be included with a better 
explanation of their purpose. 



Method 




Overall Rank 
of Usefulness 



CO 

(D 
iH 



ERIC 



O) p 
tJ 



5) U 

o 



O 



o 

•H 



O 
•H 



o 

•H 



O 
•H 



O 
• H 



O 
•H 
PQ 



II 

0) .H 



00 M iO to o»r^ in iH 

iH 

00 ^ tO 'C^J vOOCitOr^iH 

vo o iH rrooorotocsj 

iH 

lO ^ to CNJ iH 



to in 
in Tt to • • 



in 00 



in in in 
• • • 

C^J vo vo 



in 

CM 



vo 00 c^J in tj- to iH 



(D 
U 



in 

vo 



in 
in 



in 

^ c^j • to 
in 



in in in 
• in • • 
to vo to 



to CM iH 



to CM iH 



CM 1 



in in 



in 



in 
to 



in 
to 



CM 



in to 



CM 




16 



in 



to 



in 

CM 



CM 



in 
in 



-14- 



00 



in 



00 




(D 0) 
> > 
(D 0) 



1 



The data support the conclusion that the video taped teaching sequences 
are a valuable aid in the selection process, and^ that the objective suriimaTy 
sheet and pictograph be i£ more information" regarding their value is 
supplied. 

Question Tiiree 

IVhich prototypic credential provides the most confidence in 
candidate ranking? 

Hypothesis 

As more information about the candidate's teaching performance is made 
available to prospective employers, it was hypothesized that the confidence 
in ranking candidates from most desirable to least desirable would be increased. 
Referring to Table 1 this^ means that proceeding down the list of prototypic 
credentials, the confidence in ranking candidates will increase. 

?4ethod 

After obtaining a final candidate ranking, participants were asked to in- 
dicate on a 1 (not confident) to 5 (very confident) scale their confidence 
in the final rankings. For each prototypic credential group the average 
confidence rating was found and then rarfced. This ranking was compared to 
that predicted using Spearman's rank order correlation coefficient. 

Results and Discussion 

Average confidence ratings for each credential type appear in Table 9. 
Correlations between the predicted ranks and actual ranks are high and sig- 
nificantly different from zero, thus supporting the hypothesis that as more 
information about the candidate's teaching performance is provided, con- 
fidence in choosing the '-right' candidate is increased. 



Question Four 

IVhat do potential employers look for in a video tape teaching sequence? 
Hypothesis 

Jri' the science education area si5)ervisory personnel are instructed to help 
.beginning teachers by observing classroom teaching behaviors such as: 
planning for and obtaining student involvement; questioning and reacting 
techniques; presentation of abstract concepts using visual aids. Exper- 
ience and some research has shown that beginning teachers who have few 
problems in these areas usually have successful practicum experiences. 
Teaching situations were sought for inclusion in the video taped placement 
credential to illustrate a candidate's ability in these areas. One im- 
portant question, however, was: Do potential employers consider these same 
areas as important when viewing a candidate, or are other classroom behaviors 
noted? No prediction could be made regarding the answer to this question 
since little has been written on the subject • 
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Table 9 



/) 



Average confidence rating in final candidate 
ranking as a function of credential type 



Credential 
Type 



Confidence in Overall Ranking 



Biology 



Average 
Rating 



Rank 



Chemistry 

Average 
Rating 



Rank 



W 
0 
V 

wo 
w 
ov 

wov 



2.4 


2 


3.4 


3 


1.7 


1 


2.8 


i 


3.5 


3.5 


3.2 


^ 4 


3.5 


3.5 


3.7 


3.7 


^5 


4.2 


6.5 


3.9 


6 


4.1 


5 


,4.1 


7 


4.2 


6.5 



Correlation between 
predicted rank and 
actual rank 

Correlation between 
the two rankings 



.95** 



.82* 



.81* 



* Significant at .05 level 
** Significant at .01 level 
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Method 



The video portion of three different credentials were presented to six 
participants. The candidates represented by the video tapes were not the 
same as those losed previously. All candidates used in this part of the 
study were considered to be excellent teachers by the university supervisors ♦ 

One candidate was shown in an irjier» city middle school setting where students 
manifested much energy in movement and noise. One candidate, casually dressed 
was shown in a suburban high school setting teaching biology to enthusiastic 
students. Tlie third candidate was shown in a small group teaching situation 
neatly dressed in blue jeans and work shirt wearing long hair and beard. 
These video tapes were losed because it was thought that the factors described 
above would detract observers from the overall teaching performance. 

Each participant viewied the video tape alone in a room without distractions. 
Each participant was given a dictaphone and asked to think out loud while 
viewing. The recorded statements were transcribed and individual state- 
ments categorized. 



Results and Discussion . * 

Presentation of the data gathered is difficult owing to its nature. How- 
ever, five general areas were identified into* which the comments could be 
placed and are presented along with representative sanple comments. About 
fifty percent of the comments concerned teaching behaviors: "Repeats student 
answers," "Good use of blackboard," "Probing questions," "Keeps students 
interested," "Admitted he didn't know," "Tried to get students actively 
involved." Approximately ten percent of the comments concerned personal 
appearance: "The conservative element of any community would question his 
appearance," "Doesn't dress as well as most teachers, but maybe that is 
not important." It should be noted that for one participant fifty percent 
of his brief comments were directed toward the candidate's appearance and 
seemed to influence later comments negatively. (The candidate was only 
commended for his probing questions by the viewer.) 

Roughly twenty percent of the comments were made about the students: 
"Students asked the teacher a question," "Students are enthusiastic," 
"Students are engrossed in their work," "Background noise is high, but 
teacher has things under control." Five percent of participants' com- 
ments presented the viewers frame of reference. Only one participant did this 
since he recently entered educational administration from an industrial pos- 
ition and wanted to qualify his remarks. About fifteen percent of the coiranents 
were directed toward suggestions for improving the video tape: "Difficult 
to hear," "Is it possible to see student faces when answering," "I wish I 
could tell how many students were in the classroom and actually partici- 
pated in the discussion." The debriefing session conducted ffix all par- 
ticipants after the evaluation tasks were completed generally . upported 
the items noted by an individual alone. 
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From these data it seems reasonable to conclude that che participants did 
note the same types of classroom behaviors observed by science education 
supervisors. For at least one individual,- appearance and background noise 
were strong detractors away from teaching performance. Others noted the 
distractors but were not greatly influenced by them since most of the comments 
were generally favorable to the candidate. 



Recapitulation 

Addition of objective, material and video taped teaching sequences to 
educational placen.3nt credentials has met an enthusiastic response. The 
evaluative study described in this secticm has shown that the added items, 
specially the video taped teaching sequences, are useful >in selecting^ 
science teachers, and when included with the written credential materials, 
gives potential employers more confidence in their selection. From the data 
it would appear that potential employers are also concerned with teacher 
and studait classroom behaviors when viewing the video taped teaching 
.'sequences. No judgment concerning the items' ability to better represent 
actual teaching performance can be made until a more controlled study, is 
completed. From the evidence presented, the addition of these new items 
to educational placement credentials would be welcomed. 
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Additional Considerations 



The evaluation of a placement credential which represents a candidate's 

teaching ability is far more than obtaining quantitative data on. which 

to base a feasibility decision. Economic, legal and ethical issues need 

to be discussed and resolved. Since the video tape of teaching perform^ce is 

the most costly addition to the credential package, and also is the most valuabl 

in decision making, an analysis of ethical, economic and legal problems 

is needed. Space does not permit this analysis here (See Nelson (1974) for the 

analysis 1^ . Based on the analysis the- following is reconmended. 

It is the writer's position that the video tape portion of the placement 
credential be produced in^the following manner. The candidate mrasr. 
voluntarily request to have a video taped' placement credential made. 
Having, taken the initial step, the candidate is given a choice of days when • 
the. video recording eiquipment will be available and told to prepare a 
unrehearsed, 20 to 25 minute minilesson 'complete with objectives, begin 
and* end* within the time limit, and that a variety of teaching situations 
should be illustrated during that time. These teaching situations could 
be lectures, discussions, laboratory, etc. On the appointed day, the 
lesson is taught to a class chosen by the candidate. Prior to recording 
the minilesson, a visual is recorded which identifies the 'Candidate; date 
and place of graduation; teaching major and ininor; location of the 
practicum experience; and thatthe lesson was unrehearsed. The minilesson 
is recorded immediately following the introductory visual. After the lesson 
is recorded in its entirety, the candidate views the video tape. If the 
candidate approves the video taped credential, he signs a paper, accompany- 
ing the video rape, which describes the lesson^s objectives and states 
that the video tape was made voluntarily. The candidates signature would 
also authorize the placement bureau to release it to prospective employers. 
If the candidate disapproves of the video taped credential,, he would have 
one more opportunity to produce one to his satisfaction. 

The video taped credential would be owned and distributed by the place- 
ment bureau. A note would be placed ill the paper portion of the credential 
indicating to a prospective en^loyer that the candidate had made a video 
taped credential and that it was available from the placement bureau. 
Each video taped credential would be held and distributed for one year. 
After this time, the video tape would be erased and used again for another 
candidate. 

Following this procedure would eliminate questions of ethics due to jud- 
icious editing. A reduction in production tiine of two thirds would result 
and subsequently reduce the cost of prodiiction. Having the video tape made 
voluntarily and having the candidate sign a release form eliminates many 
legal issues. In some states where laws of privacy have been enacted it 
may be necessary to obtain a vrritten release from each student appearing 
in the class taught by the candidate. Consultation with a lawyer wciild 
reveal a coinrse of action. 1*^, 
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The implementation o£ these recoimiendations would add a new dimension to _ 
the selection o£ teachers. In an area of great subjoctivity, more objective 
informatioh about teaching performance is needed. This research has shown 
several ways of adding objectivity to the selection data. In addition 
to providing a letter of reference in the placement credential, a pictograph, 
competency list and video sample of teaching performance give employers 
more insight into the classroom performance of prosp^tive employees. 
All of the additions have been found to be useful, but the single most 
useful- item appears to be the sample of teaching recorded on a video tape. 
It is the writer's hope that employers will be able to say "What you see 
is Vhat you get", when utilizing these new dimensions in educational 
placement credentials. 
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EVAUIATION OF PROFESSIONAL COMPETENCIES FO R Thomas Charles Chartraw 

name of candidate 

Name o£ Write Miles A. Nelson Signature of Writer, 



Title -I' vssistant Professor. Science Education P hon e 608-263-4637 D at e 6/5/73 

School or Business University of Wisconsin A ddress 225 N. Mills Street 

; Madison, Wisconsin 53705 



Tome has been a student in my methods course and under ray direct supervision 
when he taught in a urban Wisconsin high school. His teaching ascignment 
involved chemistry and physical science. The coninents made below are oased 
on my observations of his teaching 'and working with him in the methods course. 

MASTERY . ^ * • 1 

Tom has an adequate knowledge of subject matter. He is able to present material 
in a related manner and does use a variety of resources and experiences in 
class presentations. 

PLANNING , \ 

This quite possibly is Tom's strength. He plans well and classroom routines 
are handled efficiently and with some insight. Kis long range plans include 
a variety of objectives, activities, and evaluation measures. 

TEACHING , . u u- * ' 

Tom is able to present material in a sequence relative to the subject matter. 
He moves nicely from one part of the lesson to the next as students are ready 
for it. His quest ioningN techniques are quite well developed, and they attempt 
to keep students mentally active during the lesson. Tom uses a variety of 
visual aids when appropriate. His one weakness may be that chemistry is a 
difficult subject and that students should have a difficult time in learning it. 

CLIMATE .J J u J, 

As far as administrative procedures go, he is quite well organized and handles 
these quits efficiently. *He generally uses good judgment in coping with un- 
expected behavior, aid he formulates simple but adequate clear rules for students 
to follow. He sometimes renains aloof from the students, but I have seen^ 
considerable improvement in this area. 

EVALUATION . . ^ 

He constructs tests of reasonable length and difficulty, He does use a variety 
of procedures. He is able to identify strengths and weaknesses in his own 
teaching. ^ 

PERSONAL 'CHARACTERISTICS . , . , . , c a 

Tom has energy and zest in his teaching. He has gained consiaerable confjidence 
in himself as a result of his student teaching experience. And he is willing 
to accept advice and suggestions of those in authority. 

REOJ^MENDATION 

Tom has done an unusually good job. With a little more opportunity for 
professional growth, he is almost certain to become an outstanding teacher. 
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SCHOOL OF HXJCATION FILE NO. 73-1177 

UNIVERSITY OF WISaX^ISIN-MADISON 

OBJECTIVES FOR SCIENCE ^^EIH3DS COURSE AND 'PRACTICE TEACHING 

Objectives Achieved by: Tom Chartraw E valuated by; Dr. Miles A. Nelson 

James Sime 

Location o£ Student Teaching/Intei-ship ; East High School, Madison 

During the methods course and practice teadiing, science teacher candidates at 
UW-Madison concentrate on fulfilling objectives selected from this list. Since 
time is limited, no one is expected to c(»i?>lete all of them. Proficiency must 
be demonstrated before a rating is assigned. ^Denotes objectives \Mch are 
reqidred of all students. A check in tlie (NR) column indicates no rating can be 
assigned because the objective is primarily designed for practice. All other 
objectives will be assigned ratings. 

The following rating scale is used for each objective: 

1. Achieved with unusual insight and some creativity 
2* Achieved with some insight 
3. Achieved mechanically 

SCIENCE AND CHILDREN \ 
Methods Practicum Objectives \ 

(1) iX) (3) CNR) (1) (2) iS) (NR) 

(x) 1. Reasons why science should be 

taught. 

2. Science processes: 
()()() a) l&iowledge of 
()()() C)C)() l^) Design of a research project 
()()() ()()() c) Critique student research 

projects 

3. Cognitive Development: 
*(X) a) Kiowledge of 

( ) ( ) b) Determine of child* s level of 

4. Characteristics of Discovery 

Learning 

PLANNING 

Methods Practicum Objectives 

(1) n:TrT3) m cd (2) (S) m 

1. Instructional Objectives: 
*(X) a) Identify 

*00 b) Write 

*( ) (X) ( ) c) Write for a chapter iii a text 

*( ) (X) ( ) ^) Write activities for 

objectives 

2. Unit Plan: 

*(X) ( ) ( ) ()()() a) Construct 

( ) ( 3 C ) b) Inplement 

3. Lesson Plans: 
) (X) ( ) a) Write 

(X) ( ) ( ) b) Write for the practicum 

4. Resourcefulness: 

a) Prepare a resource file 
()()() b) Utilize resources available 

O in planning lessons 
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Name Thomas Chartraw FILE NO. 73-1177 

OBJECTIVES FOR SCIENCE METHODS COURSE AI® PRACTICE TEACHING 

TEACHI?^ TACTICS 

Methods P racticun Objectives 

1. Lecture: 

a) Construct lesson plan for 

b) Successful classroom performance of 

2. Recitation: 

a) Construct lesson plan involving 

b) Successful classroom performance of 

3. Probing Discussion: 
a) Construct lesson plan for 

" b) Successful classroom performance of 
4/ Laboratory activities: 

A. Type 1 - Practice of a psychomotor 
skill: 

; a) " Construct lesson plan for 
b) Successful classroom perfoimance c 

B. Type 2 - Student velixication of a 
known concept: 

a) Construct lesson plan for 

b) Successful classroom perfoxmance 

C. Type 3 - Student discovery of coricept 

a) Construct lessort plan for 

b) Successful classroom performance 

D. Type 4 - Practice techniques of 
investigating a problem: 

a) Construct lesson plan for 

b) Successful classixxxn performance 

E. Type 5 - Student selected problem 
of study - research: 

a) Construct lesson plan for 

b) Successful classroom performance 

5. Demonstration Technique: 

a) Construct lesson plan for 

b) Successful classroom performance of 

6. Field Trips: 

a) Construct lesson plan for 

b) Successful classroom performance of 

7. Invitations to Inquiry: 
a) Construct lesson plan for 

' b) Successful classroom perfoxmance of 

8. Science and Society Discussion: 

a) Construct lesson plan for 

b) Successful classroom performance of 

9. Audio-Visual Devices: 

a) Demonstrates how to use 

b) Construct lesson plan using 

c) Successful classroom perfoxmance of 
10* Problem Solving: 

a) Construct lesson plan using 

b) Successful classroom perfoxmance of 
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Name Thomas Chartraw 



FILE NO. 73-1177 



Meti^ods 



OBJECTIVES FOR SCIENCE METHODS COURSE AND PRACTICE TEACHING 

TEACHING SKILLS 
Practicum Objectives 
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EVALUATION 



Methods 



Practicum 



a) (2) (3) (NR) (1) (2) (3) (NR) 
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() () 
( ) ( ) 
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*(X) C ) 
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Objectives 



1. Student Evaluation: 

a) Construction an evaluation device 

b) Confute test score characteristics 

c) Develop criteria for assessing a 
research report 

2. Self- Evaluation of Teaching: 

a) Video tape analysis 

b) Construct a model of a good teacher 

c) Construction a data-gathering 
instrument for self-evaluation 

d) Verbal interaction patterns 

e) Instructional patterns 

f) Student evaluation of teaching 

3. Curriculum Evaluation: 

a) Textbook analysis 

b) National Curriculum Comparison 
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Total number of student teachers/ 
interns observed 300. 



OVERALL CCMPARATIVE PERFOW^LANCE - STODENT TEACHING/ INTERNSHIP 

The following graphs sunmarize the strengths and weaknesses of the candidates 
as noted by the cooperating teacher and methods professor during the practicum. 
The categories are explained in Dr. Nelson's letter of recomnendation. 

EVALUATION BY DR. I-tiLES A. NELSON 

Excellent , 
Very Good 
Average 
Below Average 
Unsatisfactory 
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Total number of student teachers/interns 
observed 4 . 
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Nam e Thomas Chartraw File No. 75-1177 

Dat e May 1975 

THERE IS AVAILABLE FOR THIS INDIVIBUALA SHORT ■ 
(15-20 MINUTE) VIDEO TAPE OF SELECTED TEACHING 

SEQUENCES TAKEN DURING THE CANDIDATE'S FIELD / 
EXPERIENCE. THIS TAPE MLL BE HELD FOR THE 
CANDIDATE'S USE FOR ONE YEAR AFTER THE DATE 
ABOVE. THE VIDEO TAPES CAN BE MADE AVAILABLE 
TO PLAY ON MOST PLAYBACK EQUIPMENT MANUFACTURED 
AFTER 1970. IF YOU VJISH TO VIEIV THE TAPE 
EITHER AT YOUR SCHOOL AND 1«TH YOUR EQUIPMENT, 
OR AT THE UNIVERSITY OF \VISCONSIN-MADISON, 
* IVRITE TO: 

Dr. Miles A. Nelson 
Teacher Education Building 
225 North hUlls Street 
Madison, V/isconsin 55706 



I. 



ERIC 



vi 

31 



WV Chan 
H 31 

Directions: Suppose that you have a chemistr^ ^sit iok tp^f ill in your 
school. Assume a candidateL^tli an H7S7 degree will be 
paid the same as a candidate with a B.S. degree. The can- 
didates listed belo\^ have applied for the position. - Your 
task is to determine iMch candidate is most desirable, 
the next most desirable, and so forth. Us6 only the infor- 
mation supplied in order to make your decision. Place 
a 1 after the name of the candidate most desirable, 
a 2 after the next most desirable and so on until you 
have ranked all four applicants. When you have conpleted 
this task, answer the questions below. 

Thomas Chartraw 

Katherine Haller 

Thomas Kihslinger 

' Mark V/hittier 



Questions to be answered after ccHnpleting the above ranking: 

1. Rank the following with respect to usefulness to you 
in evaluating a candidate for enployment. Use a 1 
for the most useful; a 2 for the next and so on. 



Candidates personal data 
Grade Report 

' Letter of recommendation frm cooperating 
teacher 

Letter of recommendation from methods 
professor 

Letter of recommendation from university 
' supervisor 

Video tape of a lecture . ; 

Video tape of a laboratory: sessioTi;^(2^^><^<^" 
' Video tape of a laiyotT.pL(^/^ ' 



2. Was there any information in the credoitials you evaluated 
that was particularly helpful to you in arriving at a 

decision concerning the candidates. Yes ^^o 

If yes, what was this infoimation 



If no, what information would you prefer to have 



3. Did you skip portions of the video tapes? Yes 
If yes, what was your reason for doing so 
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No 



4. Ifow would you rate your confidence in the decisions made above based 
upon the information provided. 



Note confident | | | j (Very confident 
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